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Iterative  maximum-likelihood  estimation  of  the  parameters 
of  normal  populations  from  singly  and 
doubly  censored  samples 

By  11.  LEON  IL  ART  Eli  and  ALBERT  11.  MOORE 
Aerospace  Research  Laboratories  and  Air  Force  Institute  of  Technology, 

W rhjht-PattcrxoH  Air  Force  Base 


Summary 

Iterative  procedures  are  given  for  joint,  maximum-likelihood  estimation  based  on  singly 
and  doubly  censored  samples  from  a  normal  population.  The  simultaneous  equations 
yielding  the  maximum-likelihood  estimates  are  obtained.  Since  their  algebraic  solution  is 
impossible,  iterative  procedures  are  proposed  which  are  applicable  in  the  most,  general  case 
in  which  both  parameters  are  unknown  and  in  special  cases  iti  which  either  of  the  parameters 
is  known.  The  asymptotic  variances  and  covariances  arc  tabulated  for  10%  censoring 
intervals.  A  Monte  Carlo  investigation  of  the  means  and  standard  deviations  of  the 
maximum-likelihood  estimators  was  marie  for  1000  samples  from  the  standard  normal 
population  for  n  --  10  and  n  =  20.  A  comparison  was  then  made  of  best  linear  unbiased 
estimators  and  maximum-likelihood  estimators  for  n  =  10  and  n  =  20. 

1.  Introduction 

The  estimation  of  the  parameters  of  a  censored  sample  from  a  normal  populat  ion  has  been 
considered  by  many  authors,  who  have  used  several  different  methods  including  the  method 
of  least  squares  and  the  method  of  maximum  likelihood. 

Lloyd  (1952)  applied  the  theory  of  least. -squares  estimation  to  an  ordered  sample  from 
distributions  depending  on  location  and  scale  parameters  only.  Gupta  (1952)  derived  best 
linear  estimators  («.  <  10)  for  the  mean  and  variance  using  singly  censored  samples  from 
normal  populations  and  for  larger  values  of  n  derived  an  alternative  linear  estimator. 
Sarhan  A.  Greenberg  (1956,  I95S u,b)  estimated  the  mean  and  standard  deviation  of  normal 
populations  from  singly  and  doubly  censored  samples  (n  <  20)  by  the  method  of  least 
squares.  Saw  (1950)  developed  simplified  unbiased  estimators  of  the  mean  and  variance 
given  a  singly  censored  sample  from  a  normal  population  (»  <  20).  Dixon  (1957,  1960) 
derived  simplified  est  imators  of  the  mean  and  standard  deviation  for  complete  and  censored 
normal  samples  which  arc  almost  as  efficient  as  the  best  linear  estimators  (it  <  20).  Walsh 
(1956)  obtained  distribution-free  estimators  for  the  population  mean  and  variance  for 
a  rather  general  class  of  continuous  statistical  populations  using  doubly  censored  samples. 

Cohen  (1950)  used  the  method  of  maximum  likelihood  to  estimate  the  parameters  of 
normal  populations  from  singly  and  doubly  truncated  samples.  The  term  ‘truncated 
samples  ’  was  used  by  Cohen  in  a  sense  somewhat  broader  than  its  pi  esent  usage  and  included 
what  are  now  called  ‘censored  samples’.  Cohen  was  primarily  concerned,  however,  with 
Type  1  censoring  (at.  a  specified  time)  rather  than  Type  II  censoring  (when  a  specified 
number  of  failures  have  occurred).  Gupta  (1952)  found  maximum-likelihood  equations  for 
estimators  of  the  parameters  of  a  normal  population  from  a  sample  censored  from  above 
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(Typo  1 1  censoring),  mid  determined  their  asymptotic  variances  mid  covarinnoeM.  Ilalpcriu 
(1952)  proved  under  mild  regularity  conditions  tlmt  the  maximum-likelihood  estimator  of 
a  single  paramctci  from  singly  censored  samples  is  consistent,  asymptotically  normally 
distributed,  and  of  minimum  variance  for  large  samples  anil  indicated  his  results  could  be 
gonoralizod  to  several  parameters  and  more  general  censoring.  Brcakwell  (1953)  also 
obtained  maximum-likelihood  estimators  for  singly  censored  samples,  asymptotic  distribu¬ 
tions  of  the  estimators,  and  t  heir  asymptotic  biases.  Placket  t  ( 1 958)  showed  t  hat  muximum- 
lilcolihood  estimators  are  asymptotically  linear  and  that  the  best,  linear  unbiased  estimators 
are  asymptotically  normal  and  oHiciont.  I  Macke tt  computed  a  ‘lincnriznd  maximtun- 
likchhood*  loiimator  and  compared  it.  with  the  best  linear  unbiased  estimator  for  the 
standard  deviation  of  a  normal  population  from  censored  samples  (n  ^  lu).  In  three  later 
papers  Cohen  (1956,  1969,  1961)  extended  the  results  given  in  his  I960  paper.  The  present, 
paper  in  part  iiuplieat.es  the  work  of  Cohen  and  Gupta  but  extends  the  results  lor  Type  II 
censoring  to  include  maximum-likelihood  estimation  of  t  he  parameters  of  a  normal  popula¬ 
tion  from  a  doubly  censored  sample,  together  with  a  completely  computerized  iterative 
procedure,  and  mathematical  expressions  and  tables  for  asymptotic  variances  and 
covariances.  The  mathematical  formulation  for  maximum-likelihood  estimation  is  given 
in  §  2,  tho  asymptotic  variances  and  covariances  of  the  estimators  are  given  in  §  3.  A  discus¬ 
sion  of  the  iterative  procedures  for  maximum-likelihood  esi  mution  is  given  in  §4.  A  Monte 
Carlo  study  of  tho  maximum-likelihood  estimators  t  ogether  vvitli  a  comparison  with  the  best 
linear  estimator  is  given,  for  small  samples,  in  §6. 


2.  Normal  population— mathematical  formulation 
Consider  a  random  sample  of  size  n  from  a  normal  population  with  mean  /<  and  standard 
deviation  <r  and  let  Arr|ll, ....  Ar„_rj  be  the  ordorod  observations  remaining  when  the  >\ 
smallest,  observations  and  the  r„  largest  observations  havo  been  censored,  Tho  joint 
probability  density  function  of  these  order  statistics  is  given  by 


n-r, 


/(■>Vi . /<•  0-)  =  -  cr}-"* exj>  |  .  v ^  {  -  (.*  ,-  —  // )3/(2rr*)} 

where  m  =  n  -  ry  -I-  r.,. 

Thu  natural  logarithm  of  the  likelihood  function  is  given  by 
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If  m  ==  n,  i.e.  if*,  —  ra  <=  0,  these  equations  have  explicit  solutions 

f>  =  .  a  -  J[*  (■>',  -  p  ?:» j  • 

The  clot  nils  of  the  iterative  procedure  for  determining  the  maximum  likelihood  estimates 
will  V>o  given  in  §5. 


3.  ASVMt’TO'liC  VAlUANl-va  AND  oovakiancks  of  xokmai.  ks n MATO l(fi 
Gupta  (lt)f>2)  has  given  theoretical  expressions  and  a  table  for  the  asymptotic  variances 
and  covariances  of  t  in;  maximum-likelihood  est  imat  ors  of  the  parameters  of  a  normal  popu¬ 
lation  from  singly  censorcu  ftrom  above)  .sample?.  Hi?  i>  .-.oils  will  lie  extended  in  this  section 
t  o  the  case  of  doubly  censored  samples. 

The  natural  logarithm  of  the  likelihood  function  of  a  sample  of  size  ».  from  a  normal 
population  with  mean  p  and  standard  deviation  <r,  the  lowest  r,  and  the  highest  ra  sample 
values  having  been  censored,  is  given  by 

L  =  hi  -  bn  In  'In  -  m  hi  v-  V*  ’-J^L  + r,  1„  /•’(;,) e3ln  [  [  -  /-•(;„)  j,  (».  | ) 
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where  *7  =  (-hyM -/<)/«■>  =4=(-<-,1-r1~/')/"‘- 

^i)  =  fC'/(0dG  /(£..)  -  s/rL)°XP(  Ml 

and  >n  =  ?(.  —  r,-r„.  In  this  notation,  the  first  partial  derivatives  of  L  are  given  by 
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The  second  partial  derivatives  of  //  are  given  by 
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Now  lot  7j  =  r,/w,  74  =  r,/»,  ami  p  —  I  —  </i  —  </.,  —  »w/«.  As  »->og  (</,  and  (7.,  iixed),  -> 
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Tin-  elements  of  the  inftirnmtion  matrix  (multiplied  by  o'1!!)  may  be  written  an 
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The  Hay  nipt  otic  variance-oovarinnee  matrix  lbr  the  estimators  /?  mid  o’  ia  then  o-|(i,7|;«, 
where  =  fa,-, I  ''.  If  one  drops  t ho  terms  involving  from  the  equations  (3-7) -(3d*)  the 
results  agree  with  those  given  by  Gupl-a  for  the  case  of  single  censoring. 


Table  I.  Coefficients  of  <r*jn  in  asom  ptolic  variances  and  covariances  of  nuui in tandikel i/taod 
estimators  of  parameters  /t  and  a  of  nor  mal  population  from  samples  of  size  n  with  proportions 
7,  censored  from  below  and  7..  from  above 


both  parameters  unknown 
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cr  known 
a  viir(/i)/(ra 

/t  known 
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I  000000 

0-000000 
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1  000000 
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00 

0  1 
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0-1 

0-1 
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0- 000000 
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1-03501 1 

0-702002 

01 

0-2 

1  -07011 15 
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0-3 
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1-041120 
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0\ 

0-4 

1-274404 

0-370502 

1-315018 
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06 
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01 
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!nterelu\U|;mK  7,  nml  7,  leaves  variances  ami  absolute  value  of  covariance  unchanged,  Imt  ehanges 
sign  of  covariance. 


fh  iutii  e  maxi  m uiii-l i />'<  lihood  estimation  for  censored  normal  stnnph  '*  200 

The  computation  of  the  elements  rt,  of  the  information  matrix  (multiplied  by  0^,11),  ns 
gj wil  by  cquat tolls  (,'t‘7}-(!Mi),  and  the  inversion  ot  lliifi  mat  rix  to  obtain  t  lie  corUhiehtu  of 
fy’,>i  in  the  variance  covariance  matrix  were  performed  on  the  IBM  Ui20  computer.  'Die 
resulting  eoctHeients  of  ir’/n  in  vai  (/7),  cov  (/?,<>).  v«r («/■).  var(/?|o)  amt  vnr  {d  j/')  lire  {liven 
in  Table  1  fur  nil  rnmbinutions  of  7,  and  qt  w  liich  are  integral  mult  i |>lea  of  (>■  I  mid  whieh  are 
such  that  7 j  +  r/j  <  l,  and  qx  s£  7*.  Only  hull"  of  the  table  in  given  tuner  interchanging  the 
values  for  7,  and  f/4  would  produce  no  change  in  the  tabular  Values  except  that  eov  (f,&) 
changes  sign.  Values  are  given  to  six  deem  in  I  plaeea.  The  results  for  single  censoring  from 
above  ( t  i  rat  ten  lines  of  Table  1).  when  rounded  to  five  decimal  places,  agree  with  those  of 
(hipta,  except-  for  slight,  discrepancies  in  t  he  ease  7,  =«  0-0,  q„  =  <)•(). 
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The  likelihood  equal  ions  (2<3}nnd  (!?••!)  have  explicit  .solutions  only  in  the  ease  of  complete 
samples  (1/1  =  11).  For  censored  samples,  inn, „  , !‘  "i  ive  procedures  have  been  develop'  d 

for  finding  the  joint  maximum-likelihood  estimators.  Those  involve  estimating  the  para¬ 
meters,  one  at.  a  time,  in  the  cyclic  order  ft,  o',  omitting  a  parameter  if  it  is  assumed  to  be 
known.  One  starts  by  choosing  initial  estiioate(s)  for  the  unknown  parnmeter(s).  At  each 
step,  the  rule  of  false  position  (iterative  linear  interpolation)  in  used  to  determine  the  value 
of  the  parameter  then  being  estimated  which  satisfies  the  appropriate  likelihood  equation, 
in  which  the  latest  estimate  (ol‘  known  value)  of  the  other  parameter  has  been  substituted. 
Iteration  continues  until  the  results  of  successive  steps  agree  <0  within  some  assigned 
tolerance*.  Kxpcriuioo  has  shown  that  the  rate  of  convergence  is  quite  rapid  if  the  initial 
estimates  are  reasonable  and  the  amount  of  censoring  is  not  excessive. 
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SMALL  SAMIM.KS 

There  is  no  known  analyt  ic  method  of  determining  the  variances  and  covariance  of  the 
joint  distribution  of  the  maximum-likelihood  estimators  ft  and  (>  from  small  samples. 
Furthermore,  those  estimators,  while  asymptotically  unbiased,  are  known  to  be  biased  for 
small  .samples  (except  f  when  censoring  is  absent  or  .symmetric),  t  hough  analytic  expressions 
for  the  bins  are  known  only  in  tin-  ease  of  estimation  from  the  complete  sample  (mi  —  a).  In 
order  to  obtain  information  about  the  small -sample  properties  of  these  estimators,  a  Monte 
Carlo  study  was  performed  on  the  IBM  7094  computer.  I 'or  w  -»  10  and  for  11  =  20,  one 
thousand  random  sa  in  pies  of  tt  standard  normal  deviates  were  generated,  and  the  11  deviates 
in  each  sample  were  arranged  in  order  from  smallest  to  largest.  Tim  iterative  procedure 
described  in  §4  wan  used  to  compute  the  estimates  f  and  Sr,  also  «|er  and  S|/i,  from  the 
111  order  statistics  remaining  in  each  sample  after  proportions  7,  and  7.,  had  been  censored 
from  below  and  from  above,  respectively,  where  7,  and  7,,  were  taken  at  intervals  of  (>•), 
subject  to  the  restrictions  7,  ^  7.,  and  111  fs  2.  Tile  means,  variances,  and  eovnrianees  of  the 
estimates  from  1000  sampled  of  size  it  =--  10  are  given  in  Table  *2,  and  similar  results  for 
n  =  20  arc  given  in  Table  3,  There  is  no  loss  of  generality  associated  with  the  restriction 
7,  <  since  interchanging  7,  and  7.,  would  produce  no  change  in  the  expected  tabular 
values  except  that  of  reversing  the  signs  of  the  mean  of/?  and  the  covariance  of  f  and  o'. 
'I'he  rows  of  Table  2  (and  likewise  Tahir  3)  are  not  statistically  independent,  since  they  arc 
based  on  the  same  samples  (with  different  proportions  censored). 
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Tht*  following  tentative  conclusions  may  tie  drawn  from  Tables  2  and  3  .(1)  When  qi  <  q.,, 
the  estimates  ft  and  fi\tr  are  negatively  biased.  (By  symmetry,  these  estimates  are  positively 
biased  when  qi  >  qt  and  unbiased  when  qx  =  </2.)  (2)  The  estimates  a  and  a\,a  are  negatively 
biased  regardless  of  the  relative  magnitude  of  and  q2-  (3)  The  bias  in  estimating  either 
parameter  is  much  smaller  when  the  other  parameter  is  known  than  it  is  when  both  para¬ 
meters  are  being  estimated  simultaneously.  (4)  The  bias  of  <r  {ft  unknown)  is  approximately 
equal  to  —  1  jm. 

It.  would  be  desirable  to  compare  the  variances  of  /<  and  tr  from  samples  of  sixes  10  and  20 
with  the  values  which  one  would  obtain  by  substituting  a  =  10  and  n  =  20  in  the  asymptotic 
values  given  in  Table  1,  as  well  as  wit  h  the  variances  of  the  best  linear  unbiased  estimators 
ft*  and  cr *.  Direct  comparison  of  variances  of  estimators  is  appropriate,  however,  only  when 
all  the  estimators  arc  unbiased.  In  order  to  compensate  for  the  hiiis  in  the  maximum- 
likelihood  estimators,  the  mean  square  errors  of  ft,  <x,  ft\a,  and  a\/t  were  computed.  These 
were  compared  with  the  variances  of  the  best  linear  unbiased  est  imators  given  by  Sarhan  & 
Greenberg  (1962,  Table  10C  2)  and  with  the  variances  of  the  maximum -likelihood  esti¬ 
mators  given  by  the  asymptotic  formula,  which  were  obtained  by  dividing  by  n  the  values 
given  in  Table  1.  The  results  are  shown  in  Tables  4  and  5  from  which  the  following  tentative 
conclusions  may  be  drawn :  ( 1 )  The  precision  of  the  maximum-likelihood  estimator  ft,  when 
proper  allowance  is  made  for  bias,  closely  approximates  that  predicted  by  the  aysrnptotic 
formula  for  tire  variance  of  ft,  even  for  m  as  small  as  2,  except  in  eases  of  strongly  asym¬ 
metric  censoring.  (2)  The  precision  of  the  maximum-likelihood  estimator  a,  when  proper 
allowance  is  made  for  bias,  closely  approximates  that  predicted  by  the  asymptotic  formula 
for  the  variance  of  <r,  except  when  m  is  quite  small  and/or  censoring  is  strongly  asymmetric. 
(3)  Maximum-likelihood  estimators  tend  to  be  somewhat  more  precise  than  best  linear 
unbiased  estimators.  The  difference  is  greatest  for  estimates  of  //  in  cases  of  strongly  asym¬ 
metric  censoring  and  for  estimates  of  <r  when  m  is  small  and/or  censoring  is  strongly 
asymmetric. 

Approximate  corrections  for  the  bias  of  the  maximum-likelihood  estimators  ft,  <r,  ft)<r,  and 
<r|<r  for  n  =  10  and  n  —  20  can  be  made  by  use  of  the  means  found  in  tho  Monte  Carlo  study 
and  recorded  in  Tables  2  and  3. 

The  authors  are  indebted  to  the  referee  for  suggesting  a  number  of  improvements  in  the 
original  draft  of  this  paper. 
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